A semiconductor memory device is under developing, which includes three-dimensionally arranged memory cells. For example, a NAND type memory device includes a plurality of word lines stacked on a base layer and semiconductor layers each provided in a memory hole extending through the word lines. The memory cells are provided at portions respectively, where the semiconductor layers intersect the word lines. In order to increase the memory capacity of such a memory device, it is desirable to increase the number of stacked word lines and increase the density of semiconductor layers in a surface of the word line. Thus, the memory holes are formed deeper in the stacking direction of the word lines, and the diameter thereof is miniaturized. As a result, the uniformity of diameters in the memory holes is lost in the stacking direction, and for example, the memory cell characteristics may have many variations.